Search for: All records

A high-quality hydrographic observational database is essential for ocean and climate studies and operational applications. Because there are numerous global and regional ocean databases, duplicate data continues to be an issue in data management, data processing and database merging, posing a challenge on effectively and accurately using oceanographic data to derive robust statistics and reliable data products. This study aims to provide algorithms to identify the duplicates and assign labels to them. We propose first a set of criteria to define the duplicate data; and second, an open-source and semi-automatic system to detect duplicate data and erroneous metadata. This system includes several algorithms for automatic checks using statistical methods (such as Principal Component Analysis and entropy weighting) and an additional expert (manual) check. The robustness of the system is then evaluated with a subset of the World Ocean Database (WOD18) with over 600,000in-situtemperature and salinity profiles. This system is an open-source Python package (named DC_OCEAN) allowing users to effectively use the software. Users can customize their settings. The application result from the WOD18 subset also forms a benchmark dataset, which is available to support future studies on duplicate checks, metadata error identification, and machine learning applications. This duplicate checking system will be incorporated into the International Quality-controlled Ocean Database (IQuOD) data quality control system to guarantee the uniqueness of ocean observation data in this product. 

Abstract Earth’s energy imbalance (EEI) is a fundamental metric of global Earth system change, quantifying the cumulative impact of natural and anthropogenic radiative forcings and feedback. To date, the most precise measurements of EEI change are obtained through radiometric observations at the top of the atmosphere (TOA), while the quantification of EEI absolute magnitude is facilitated through heat inventory analysis, where ~ 90% of heat uptake manifests as an increase in ocean heat content (OHC). Various international groups provide OHC datasets derived from in situ and satellite observations, as well as from reanalyses ingesting many available observations. The WCRP formed the GEWEX-EEI Assessment Working Group to better understand discrepancies, uncertainties and reconcile current knowledge of EEI magnitude, variability and trends. Here, 21 OHC datasets and ocean heat uptake (OHU) rates are intercompared, providing OHU estimates ranging between 0.40 ± 0.12 and 0.96 ± 0.08 W m⁻²(2005–2019), a spread that is slightly reduced when unequal ocean sampling is accounted for, and that is largely attributable to differing source data, mapping methods and quality control procedures. The rate of increase in OHU varies substantially between − 0.03 ± 0.13 (reanalysis product) and 1.1 ± 0.6 W m⁻² dec⁻¹(satellite product). Products that either more regularly observe (satellites) or fill in situ data-sparse regions based on additional physical knowledge (some reanalysis and hybrid products) tend to track radiometric EEI variability better than purely in situ-based OHC products. This paper also examines zonal trends in TOA radiative fluxes and the impact of data gaps on trend estimates. The GEWEX-EEI community aims to refine their assessment studies, to forge a path toward best practices, e.g., in uncertainty quantification, and to formulate recommendations for future activities.